In a database system that supports both streaming and batch processing of data (e.g., real-time display of collected/received information such as stocks as well as historical view of the same information collected over a period of time), query results of a streaming processing and batch processing can be inconsistent. The reason for this inconsistency is that two separate data paths exist for streaming processing and batch processing due to the requirement of faster access to the incoming data for streaming processing while such requirement is less stringent for batch processing of the incoming data.
In this system, there is no guarantee that the incoming data that is used for streaming processing along the streaming data path would be the same as the incoming data that is used for batch processing along the batch data path. This is due to the fact that the data for batch processing is written to a database. This writing of the data to the database may fail due to a number of reasons (e.g., due to redundancy check failure performed on the incoming data). Therefore, the data that is stored in the database and ultimately used for batch processing will be different from the data used for streaming processing. Improvements are needed to ensure consistency between the data used for streaming processing and the data used for batch processing.